1. Field of the Invention
The present invention relates to a 3-dimensional virtual environment communication system, in which a number of users are connected to a central control equipment through communication lines to share the 3-D virtual environment created by 3-D computer graphics.
2. Description of the Related Art
Conventionally, there have been available television conferencing systems to enable a plurality of users located in different locations to confer with each other under the control of a central control equipment, or shared 3-D virtual environment communication systems based on 3-dimensional computer graphics technique (referred to as 3-D computer graphics hereinbelow). Television conferencing systems are used only for a very specialized purpose of conducting discussions among the users, and in the 3-D virtual shared environment communication based on 3-D computer graphics, there are systems that are based on text communications through chat-type format.
It has been difficult to introduce television conferencing systems based on central control equipment, because of the high cost of the equipment itself and communications cost associated with the conference. Also, although the 3-D virtual shared environment communications based on 3-D computer graphics have an advantage that the user is free to move about and the methodology is intuitive so that the system is user-friendly, however, because of the degree of user mobility is too high, the system is not suitable for use in the specialized purpose of conferencing, when it is desirable to face the person in conversation with or to have many users to confer with each together while viewing facial expressions of other participants on screen in real-time.
Therefore, it is important in those conferencing systems used for business purposes to hear each other""s voices as well as to view the actual expressions on the users faces, and from the viewpoint of wider applicability of the communication system, it is desirable to increase the flexibility of the communication system.
It is a first object of the present invention to provide a system and a method for communication to enable a plurality of participants to share a 3-D virtual environment so that real-time facial expressions of other participants can be seen and voices of other participants heard on one""s own terminal, thus making the system applicable to a virtual conferencing communication system.
Also, a second object is to realize a television conferencing capability that has been limited only to televising technology, by utilizing system and method customized to a conferencing format, and the system and method can be executed readily by providing computer-readable recorded medium containing executable application programs that enable to realize the communication system in a more flexible 3-D virtual environment that can be shared by a number of participants by displaying users"" real-time facial images and reproducing voices on real-time on user terminals.
To realize the object, the present invention provides a method for displaying a three-dimensional shared virtual environment comprising the steps of: displaying a first display state of a virtual environment, generated by using a three-dimensional computer graphics technique, to be displayed in individual user terminals of a plurality of users so that a computer generated graphic object representing a subject user may be moved in accordance with instructions give by the subject user; and displaying a second display state of the virtual environment so that computer generated graphic objects representing the plurality of users are always located in constant positions within the virtual environment.
The method may further include a step of selecting either the first display state or the second display state according to instructions given by a user so as to display a selected virtual environment on the user terminal.
Accordingly, the method has an advantage that motion of the users within the virtual environment is natural and can be accepted intuitively, and when necessary, other users whose images are assigned to fixed locations may be reference for conferencing purpose, therefore, the system can be made quite flexible in meeting users"" needs.
It is preferable that the method further includes the steps of: displaying pasted images of individual users by pasting corresponding real-time facial images on respective computer generated graphic objects representing the individual users so that, when displaying the second display state on a user terminal, front views of the computer generated graphic objects and corresponding pasted images are displayed. It is also preferable that a plurality of motion states for expressing intentions of a user represented by the computer generated graphic object be stored, so that a motion state selected uniquely according to a user instruction is pasted onto the computer generated graphic object and a pasted image is displayed.
The present invention also provides a three-dimensional shared communication system connecting a plurality of user terminals to central control means through communication lines, wherein each of the user terminals is comprised by: image inputting means for inputting real-time images of a user; voice audio inputting means for inputting voice data of the user; motion information inputting means for inputting motion information of the user within a virtual environment described by three-dimensional computer graphics; action indicating means for indicating actions of a computer generated graphic object representing the user to be displayed on terminal display means of other users; simple program outputting means for outputting a pre-determined simple program according to an instruction given by the action indicating means; user information input/output means for broadcasting received real-time image data, voice data, motion data and simple program data to other user terminals through the central control means, and receiving real-time image data, voice data, motion data and simple program data of the other users through the central control means; voice audio reproduction means for reproducing voices of the other users received by the user information inputting/outputting means; virtual environment control means for generating information required for displaying computer generated graphic objects representing the other users according to motion information and the simple program received by the user information input/output means from other user terminals, and controlling images displayed in such a way that the computer generated graphic objects are always displayed in assigned positions and facing frontward; and virtual environment display means for pasting real-time facial images of the other users in the computer generated graphic objects according to information generated by the virtual environment control means and displaying a combined image of the virtual environment and the pasted images.
It should be noted that the computer-generated graphic objects may be represented in two-dimensions or in three-dimensions. The motions of computer-generated graphic objects indicate users"" intentions, such as xe2x80x9craise handxe2x80x9d, xe2x80x9cagreexe2x80x9d, and xe2x80x9crejectxe2x80x9d. Front view of the computer-generated graphic object means that the pasted facial images are displayed so as to face the other users viewing the virtual environment on the monitor screen.
According to the present communication system, user terminals show virtual environment produced by computer graphics as well as facial images of other users who are participating in the virtual environment, which are pasted into computer-generated graphic objects corresponding to these users, and displayed along with their voices. It is also possible to express gestures of the participants through the computer-generated graphic objects. The system also allows the motion information input section to notify any changes in the position of the users so that, in spite of the motion of the users, computer-generated graphic objects representing the users remain in assigned positions and the front view of their facial images are displayed to the viewer of the virtual environment, which facilitates the communication process.
The present invention provides a three-dimensional shared communication system connecting a plurality of user terminals to central control means through communication lines, wherein the central control means stores information defining a plurality of types of virtual environments with individual addresses, and having at least one information defining a virtual conferencing environment; and each of the plurality of user terminals is comprised by: address specifying means for specifying addresses of information defining the virtual environments; image inputting means for inputting real-time images of a user; voice audio inputting means for inputting voice data of the user; motion information inputting means for inputting motion information of the user within a virtual environment described by three-dimensional computer graphics; action indicating means for indicating actions of a computer generated graphic object representing the user to be displayed on terminal display means of other users; simple program outputting means for outputting a pre-determined simple program according to an instruction given by the action indicating means; user information input/output means for broadcasting received real-time image data, voice data, motion data and simple program data to other user terminals through the central control means, and receiving real-time image data, voice data, motion data and simple program data of the other users through the central control means; voice audio reproduction means for reproducing voices of the other users received by the user information inputting/outputting means; virtual environment control means for generating information required for displaying the virtual environment according to information defining the virtual environment and motion information received by the user information input/output means from user terminals logging in to the virtual environment, and generating information required to display computer generated graphic objects representing the other users according to motion information and the simple program received; virtual environment display means for pasting real-time facial images of the other users in the computer generated graphic object according to information generated by the virtual environment control means, and displaying a combined image of the virtual environment and the pasted images, and wherein, when an address of information defining the virtual conferencing environment is specified by the address specifying means, the virtual environment control means control displays so that the computer generated graphic objects always remain in assigned locations and facing frontward.
According to the communication system presented above, in addition to the effects produced by the communication system disclosed in claims 1xcx9c3, it enables the user to readily switch between the virtual conferencing environment and another type of virtual environment to provide a more flexible communication system.
The present invention provides a virtual conferencing system for conducting a conference among users of the user terminals using one communication system chosen from any of the three-dimensional shared virtual environment communication systems presented above.
The present invention relates also to a method for communicating among a plurality of user terminals in a three-dimensional virtual environment shared through communication lines connected to each user terminal so as to exchange data between any one user terminal and other user terminals through central control means, comprising the steps of: inputting real-time images of a user of one user terminal produced by an imaging device provided in each user terminal; inputting voice of the user obtained by a sound recording device provided in each user terminal; inputting motion information of the user within the virtual environment described by three-dimensional computer graphics produced by a motion information input device provided in each user terminal; inputting action instructions of a computer generated graphic object, representing the user to be displayed in other user terminals; outputting a pre-determined simple program from the central control means in response to motion instructions input from the one user terminal; broadcasting real-time facial image data, voice audio data, motion information data, and the simple program, output from the one user terminal, to other user terminals through the central control means, and receiving real-time facial image data, voice audio data, motion information data of other users and the simple program, output from the other user terminals, in the one user terminal through the central control means; reproducing voices of other users received from the other user terminals at the one user terminal; generating information required for displaying computer generated graphic objects representing the other users using the user information input/output means according to motion information and the simple program, and controlling image orientations in such a way that the computer generated graphic objects are always displayed in assigned locations and facing frontward in the virtual conferencing environment; and pasting real-time facial images of the other users in the computer generated graphic objects according to generated information and displaying a combined image of the virtual environment and the pasted images.
The present method further includes the steps of: obtaining information related to users participating in the virtual environment being managed by the central control means; judging whether there has been a change in a number of users participating, based on information obtained; and altering computer generated graphic objects to be displayed in the virtual environment according to judged results.
Further, the present invention provides a method for communicating among a plurality of user terminals in a three-dimensional virtual environment shared through communication lines connected to each user terminal so as to exchange data between any one user terminal and other user terminals through central control means, comprising the steps of: inputting real-time images of a user of one user terminal produced by an imaging device provided in each user terminal; inputting voice of the user obtained by a sound recording device provided in each user terminal; inputting motion information of the user within the virtual environment described by three-dimensional computer graphics produced by a motion information input device provided in each user terminal; inputting action instructions of a computer generated graphic object, representing the user to be displayed in other user terminals; outputting a pre-determined simple program from the central control means in response to motion instructions input from the one user terminal; broadcasting real-time facial image data, voice audio data, motion information data, and the simple program, output from the one user terminal, to other user terminals through the central control means, and receiving real-time facial image data, voice audio data, motion information data of other users and the simple program, output from the other user terminals, in the one user terminal through the central control means; reproducing voices of other users received from the other user terminals at the one user terminal; specifying addresses to correspond to at least one of a plurality of types of virtual environments, including a virtual conferencing environment, identified by own addresses stored in the central control means; generating information necessary for displaying a relevant virtual environment according to information defining the relevant virtual environment identified by a specified address and motion information received from other user terminals logging in to the relevant virtual environment, and generating information necessary for displaying computer generated graphic objects representing the other users according to received information and the simple program; pasting real-time facial images of the other users in the computer generated graphic objects according to generated information and displaying a combined image of the virtual environment and the pasted images; and controlling orientation of images in such a way that the computer generated graphic objects are always displayed in assigned positions and facing frontward in the virtual conferencing environment.
The present invention also provides a computer-readable recording medium containing an application program for executing a method of communicating in a three-dimensional shared virtual environment using each of a plurality of user terminals, connected to central control means through communication lines, by means of a common virtual environment shared by the plurality of user terminals, comprising the steps of:
inputting real-time images of a user of one user terminal produced by an imaging device provided in each user terminal;
inputting voice of the user obtained by a sound recording device provided in each user terminal;
inputting motion information of the user within the virtual environment described by three-dimensional computer graphics produced by a motion information input device provided in each user terminal;
inputting action instructions of a computer generated graphic object, representing the user to be displayed in other user terminals;
outputting a pre-determined simple program from the central control means in response to motion instructions input from the one user terminal;
broadcasting real-time facial image data, voice audio data, motion information data, and the simple program, output from the one user terminal, to other user terminals through the central control means, and receiving real-time facial image data, voice audio data, motion information data of other users and the simple program, output from the other user terminals, in the one user terminal through the central control means;
reproducing voices of other users received from the other user terminals at the one user terminal;
generating information required for displaying computer generated graphic objects representing the other users using the user information input/output means according to motion information and the simple program, and controlling image orientations in such a way that the computer generated graphic objects are always displayed in assigned locations and facing frontward in the virtual conferencing environment; and
pasting real-time facial images of the other users in the computer generated graphic objects according to generated information and displaying a combined image of the virtual environment and the pasted images.
In the computer-readable recording medium presented above, the application program may further contain the steps of: obtaining information related to users participating in the virtual environment being managed by the central control means; judging whether there has been a change in a number of users participating, based on information obtained; and altering computer generated graphic objects to be displayed in the virtual environment according to judged results.
The present invention further provides a computer-readable recording medium containing an application program for executing a method of communicating in a three-dimensional shared virtual environment using each of a plurality of user terminals, connected to central control means through communication lines, by means of a common virtual environment shared by the plurality of user terminals, comprising the steps of: inputting real-time images of a user of one user terminal produced by an imaging device provided in each user terminal; inputting voice of the user obtained by a sound recording device provided in each user terminal; inputting motion information of the user within the virtual environment described by three-dimensional computer graphics produced by a motion information input device provided in each user terminal; inputting action instructions of a computer generated graphic object, representing the user to be displayed in other user terminals; outputting a pre-determined simple program from the central control means in response to motion instructions input from the one user terminal; broadcasting real-time facial image data, voice audio data, motion information data, and the simple program, output from the one user terminal, to other user terminals through the central control means, and receiving real-time facial image data, voice audio data, motion information data of other users and the simple program, output from the other user terminals, in the one user terminal through the central control means; reproducing voices of other users received from the other user terminals at the one user terminal; specifying addresses to correspond to at least one of a plurality of types of virtual environments, including a virtual conferencing environment, identified by own addresses stored in the central control means; generating information necessary for displaying a relevant virtual environment according to information defining the relevant virtual environment identified by a specified address and motion information received from other user terminals logging in to the relevant virtual environment, and generating information necessary for displaying computer generated graphic objects representing the other users according to received information and the simple program; pasting real-time facial images of the other users in the computer generated graphic objects according to generated information and displaying a combined image of the virtual environment and the pasted images; and controlling orientation of images in such a way that the computer generated graphic objects are always displayed in assigned positions and facing frontward in the virtual conferencing environment.